1. Field of the Invention
The present invention relates to a video transmission device and its method and more particularly, is suitably applied to a digital broadcasting system.
2. Description of the Related Art
A digital broadcasting system for multiplexing coded video and audio data to transmitting them through a satellite circuit has been proposed and become practical use. In this digital system, video data and also audio data are encoded, and by packetizing and multiplexing the coded video and audio data per fixed data unit, transport data (hereinafter, referred to as a transport stream) is formed and transmitted through such as a satellite wireless circuit.
Here, the digital broadcasting system will be described with reference to FIG. 1. As shown in FIG. 1, a digital broadcasting system 1 is roughly comprised of a transmission device 2 to be set in a broadcasting station, a receiving device 3 and a television receiver 4 to be installed in each house. Transport stream data to be transmitted from the transmission device 2 is received by the receiving device 3, and by sending out this to the television receiver 4, broadcasting video and audio can be watched.
As shown in FIG. 2, in the transmission device 2, firstly, video data S1 to be transmitted is entered into a video encoder 5, and audio data S2 accompanying with the video data SI is entered into an audio encoder 6. The video encoder 5 successively encodes the entered video data S1 based on the MPEG2 video encoding system described in the International Standards Regulation ITU-T Recommendation H.262 ISO/IEC13818-2, and outputs a stream of the resulting coded video data S3 (generally, this is called as an elementary stream (ES)) to a packetizer 7A. In the case of encoding the video data S1, the video encoder 5 applies an encoding processing with video data for several pictures, which are called as group of picture (GOP), as a coding unit, and by adding a header consisting of each parameter information at the time of encoding to the head of the coded video data, forms coded video data S3. Thus, a decoding processing can be conducted based on the parameter information of the header at the time of decoding.
As shown in FIGS. 3A and 3B, the packetizer 7A, by connecting a fixed number of ES packets of coded video data S3 encoded per GOP and adding a packetized elementary stream (PES) header to the head of the packet, PES-packets the coded video data S3 and outputs the PES-packetized coded video data S4 to a multiplexer 8. Note that, data of the PES header consist of time information when reproducing and outputting the video data included in the PES packet at the receiving side (generally, called as presentation time stamp (PTS)), identification information showing the type of elementary stream included in the PES packet (generally, called as a stream ID), and the like. In this connection, such time information PTS is provided in the PES header, so that audio and video can be synchronized and reproduced at the receiving side.
On the other hand, the audio encoder 6 encodes the input audio data S2 based on an audio encoding system described in the International Standards Regulation ITU-T Recommendation H.262 ISO/IEC13181-3 and outputs the resultant coded audio data S5 to a packetizer 7B. Note that, the audio encoder 6 encodes the audio data S2 per fixed number of samples called as an audio frame in the same manner as in the case of video data shown in FIG. 2, and by adding a header comprising parameter information at the time of encoding to the head of the coded audio data, forms an ES-packetized coded audio data S5.
Moreover, the packetizer 7B, by connecting a fixed number of ES packets of the coded audio data S5 encoded per audio frame and adding the PES header to the head of the packet, PES-packetizes the coded audio data S5 and outputs the PES-packetized coded audio data S6 to the multiplexer 8. The data of the PES header are approximately the same as in the case of video.
The multiplexer 8 transport-stream (TS)-packetizes the PES-packetized coded video data S4 and coded audio data S6 by dividing them per fixed data unit respectively, and by switching and outputting the TS packets of the video and audio at established intervals, multiplexes them to form a transport stream S7.
More specifically, as shown in FIGS. 5A, 5B, 6A and 6B, the multiplexer 8,,by dividing each PES packet of the coded video data S4 and coded audio data S6 per 184 bytes and adding a TS header of 4 bytes to each of their heads, forms a TS packet. In this case, the TS header is formed of synchronization information and packet identification information PID showing the type of data included in the TS packet. However, the data part of 184 bytes is not only comprised of PES data but in practice, also additional information is added and these additional information and PES data are adjusted to become 184 bytes altogether. Such data adjusting processing is generally called as a stuffing processing. In this connection, even when PES data are not enough, the stuffing processing is conducted so that data part becomes 184 bytes by adding dummy byte. Then, the multiplexer 8 forms a transport stream S7 by multiplexing thus formed video TS packets onto thus formed audio TS packets.
In this connection, since proper packet identification information PID is added to the multiplexed video and audio TS packet per data, the receiving side can separate the TS packets based on the packet identification information PID to decode video and audio from one multiplexed transport stream.
Thus, the transport stream S7 comprised of video and audio TS packets is transmitted to the succeeding transmission unit 9. The transmission unit 9, after forming a transmission signal by applying the modulation processing to the transport stream S7 for transmitting, forms a transmission signal S8 by applying a frequency conversion processing to the transmission signal of a stated band and transmits this toward the satellite (not shown) via an antenna 10.
A transmission signal S8 relayed via the satellite is received by the receiving device 3 shown in FIG. 1 and the receiving device 3 conducting a reverse processing of the transmission device 2, so that the transmission signal S8 is decoded to the original video and audio data. More specifically, the receiving device 3 applies the frequency conversion processing and demodulation processing to the received signal to restore the transport stream S8 transmitted from the transmission device 2. And then, the receiving device 3 separates video TS packets and audio TS packets based on the packet identification information PID and takes out coded video data and coded audio data from each TS packet separated, and applying the decoding processing to these data, restores video data and audio data transmitted. Thus, these restored video data and audio data are transmitted to the television receiver 4, so that video and audio of the program sent from the transmitter 2 can be watched.
In the foregoing explanation, the transmission device 2 transmits video and audio data forming one program. However, in practice, the transmission device 2 transmits video and audio data of the plural number of programs in the form of one transport stream S8 by TS-packetizing and multiplexing video and audio data of plural programs. Furthermore, in the case where text data are attached to the video data, the text data can also be transmitted in the form of one transport stream S8 by being TS-packetized and multiplexed. In this connection, if the plural programs are multiplexed, the receiving side extracts TS packets of a desired program based on the packet identification information PID added to the TS packets and thus, the video and audio data of the desired program can be restored.
At this point, the processing in the case of multiplexing the plural programs will be described in detail in the following paragraphs. Since the receiving side extracts the TS packets of the desired program based on the packet identification information PID as described above, it is necessary to know the packet identification information PID added to the TS packet of video and audio data of each program. Accordingly, the transmission device 2 TS-packetizes the program information showing the packet identification information PID added to the TS packet of each program, and multiplexes and transmits them onto the video and audio TS packets.
In this case, the program information to be transmitted is formed hierarchically, and consists of program information of higher hierarchy called as program association table (PAT) and program information of lower hierarchy called as program map table (PMT). As shown in FIG. 7, the higher hierarchy program information PAT is the information to show the packet identification information PID of TS packet in which the program information PMT formed per program is stored, that is, the information like a table of contents on the program information PMT. Referring to the program information PAT, it is possible to know in which TS packet the program information PMT of each program is stored. For example, it is clear that the program information PMT having the program number xe2x80x9c0xe2x80x9d is stored in the TS packet having the packet identification information PID xe2x80x9cNxe2x80x9d. Note that, xe2x80x9c0xe2x80x9d is added to the TS packet storing the program information PAT as the packet identification information PID. Thus, the receiving side, extracting the TS packet whose packet identification information PID is xe2x80x9c0xe2x80x9d from the transport stream received and decoding it, can obtain the program information PAT.
On the other hand, as shown in FIG. 8, the program information PMT is the information to show the packet identification information PID of TS packet in which each data forming the program is stored, per program and is the detailed program information formed per program. Accordingly, by referring to the program information PMT, it is possible to know in which TS packet each data forming the program is stored. For example, it is clear that the video data having the program number xe2x80x9cXxe2x80x9d is stored in the TS packet having the packet identification information PID xe2x80x9cXVxe2x80x9d, and the audio data having the program number xe2x80x9cXxe2x80x9d is stored in the TS packet having the packet identification information PID xe2x80x9cXAxe2x80x9d. Further, the text data having the program number xe2x80x9cXxe2x80x9d is stored in the TS packet having the packet identification information xe2x80x9cXDxe2x80x9d, and the clock information having the program number xe2x80x9cXxe2x80x9d is stored in the TS packet having the packet identification information PID xe2x80x9cXCxe2x80x9d.
At this point, an algorithm in the case of selecting a desired program in the receiving device 3 based on the program information PAT and PMT will be described below with reference to FIG. 9. First, in the receiving device 3, after turning the power source ON, the TS packet having the packet identification information PID xe2x80x9c0xe2x80x9d is extracted from the transport stream received to obtain the program information PAT. At this point, if a request to display the program having the program number xe2x80x9cXxe2x80x9d is entered by the user, the receiving device 3, referring to this program information PAT, investigates the packet identification information PID of TS packet in which the program information PMT having the program number xe2x80x9cXxe2x80x9d is stored. In this case, since the TS packet in which the program information PMT having the program number xe2x80x9cXxe2x80x9d is stored has the packet identification information PID xe2x80x9cXXxe2x80x9d, the receiving device 3 extracts the TS packet having the packet identification information PID xe2x80x9cXXxe2x80x9d from the transport stream received to obtain the program information PMT concerning the program number xe2x80x9cXxe2x80x9d.
The receiving device 3, by referring to the program information PMT obtained, can grasp the TS packet in which each data associated with the program number xe2x80x9cXxe2x80x9d is stored. Thus, the receiving device 3 obtains and decodes coded video data having the program number xe2x80x9cXxe2x80x9d by extracting the TS packet having the packet identification information PID xe2x80x9cXVxe2x80x9d from the transport stream received. Simultaneously, the receiving device 3, by extracting the TS packet having the packet identification information PID xe2x80x9cXAxe2x80x9d from the transport stream, obtains and decodes the coded audio data having the program number xe2x80x9cXxe2x80x9d. Moreover, in tandem with the video and audio decoding processing, the receiving device 3, by extracting the TS packet having the packet identification information PID xe2x80x9cXDxe2x80x9d from the transport stream received, obtains and decodes text data on the program number xe2x80x9cXxe2x80x9d. Simultaneously, by extracting the TS packet having the packet identification information PID xe2x80x9cXCxe2x80x9d, the receiving device 3 obtains clock information on the program number xe2x80x9cXxe2x80x9d and generates a recovery clock to become the reference for decoding processing and display-output based on the clock information. In this connection, the recovery clock is necessary for decoding video data and audio data, in the strict sense, the generating processing of the recovery clock is to be started first.
Accordingly, the receiving device 3, firstly, obtains the program information PAT which is equivalent to a table of contents, and then referring to the program information PAT, obtains the program information PMT showing the details of a program requested by a user and referring to the program information PMT, obtains coded video data and coded audio data concerning the requested program. With this arrangement, even in the case where multiple programs are multiplexed in one transport stream, the requested program can be reproduced and sent out.
At this point, the generating processing of recovery clock described above will be explained more concretely. The generating processing of the recovery clock is generally called as PCR recovery and is for recovering the clock used in the transmission device 2, in the receiving device 3. In the transmission device 2, the encoding processing for video data and audio data are conducted in synchronism with 27 MHz clock. In the receiving device 3, the same clock as the clock at the time of the encoding processing is reproduced and the decoding processing and the display-output are conducted at the timing specified by the transmission device 2, in order to correctly reproduce video data and audio data without excess or lack of data.
Accordingly, in the digital broadcasting system 1, clock synchronization is to be conducted by the processing described as follows in the receiving side. As shown in FIG. 10, a counter 11 called as a system time clock (STC) is provided in the transmission device 2. This counter (STC) 11 is formed of 33-bit counter, and by conducting the counting operation with 27 MHz clock CLK1, which is generated in the clock generator 12 and becomes the reference clock for the encoding processing, can count 24 hours. In the transmission device 2, making the count value of the counter (STC) 11 as clock information PCR, the packetizer 7C TS-packetizes at a fixed cycle. The multiplexer 8 multiplexes and transmits the TS packet consisting the clock information PCR together with other TS packets such as video and audio. Note that, the packet identification information PID of the clock information to be regulated by the aforementioned program information PMT is added to the TS packet including the clock information PCR. Accordingly, the receiving device 3, by referring to the program information PMT, can extract the TS packet in which the clock information PCR on that program is included.
The receiving device 3, referring to the program information PMT described above, extracts the TS packet in which the clock information PCR is included. In the example shown in FIG. 8, since the packet identification information PID of the clock information is xe2x80x9cXCxe2x80x9d, the TS packet having the packet identification information PID xe2x80x9cXCxe2x80x9d is extracted by the demultiplexer 13. The receiving device 3 has a counter (STC) 15 which conducts the counting operation at 27 MHz clock CLK2 generated in the clock generator 14, and sets the clock information PCR included in the extracted TS packet to the counter (STC) 15 as an initial value, and makes the counter (STC) 15 to count operate based on the clock CLK2. Then, obtaining the difference between the clock information transmitted at a fixed cycle and the count value of the counter (STC) 15, and by controlling the operation of the clock generator 14 by a phase locked loop (PLL) 16 based on the difference, the receiving device 3 forms a clock CLK2 of which phase and frequency completely conform with that of the clock CLK1 generated in the transmitting side. Thus, by transmitting the clock information PCR from the transmission device 2 and controlling the operation of the clock generator 14 based on the clock information PCR in the receiving device 3, the clock CLK2 having the same frequency and phase as the transmitting side can be reproduced in the receiving device 3 and the operation of the counter (STC) 15 can be synchronized with the operation of the transmitting side.
In the transmission device 2 of the transmitting side, in the case of forming the PES packet, the time information PTS to regulate the timing of reproduction and output of video or audio data contained in the PES packet is added. And the decoding processing is conducted at the timing based on the time information PTS, in order to synchronize video and audio without excess or lack of data. Accordingly, the receiving device 3 takes out the time information PTS added when PES packetizing, from the TS packet received, compares the time information PTS to the value of its own counter (STC) 15, and starts the decoding processing at the timing when these values coincide.
For example, in the case of decoding video data of the program in the example shown in FIG. 8, firstly the demultiplexer 13 takes out the TS packet having the packet identification information PID xe2x80x9cXVxe2x80x9d from the transport stream received, and restores PES packet from the TS packet and transmits this to the decoder 17, as shown in FIG. 11. Moreover, the demultiplexer 13 transmits the PES packet to the PTS receiving unit 18. The PTS receiving unit 18 extracts time information PTS from the PES packet and transmits this to a time comparator 19. The count value of the counter (STC) 15 is also supplied into the time comparator 19, and the time comparator 19 compares the count value of the counter (STC) 15 to the value of time information PTS supplied from the PTS receiving unit 18. If these count values coincide, the timing information showing the decoding start timing is transmitted to the decoder 17. With this arrangement, in the decoder 17, by starting to decode the PES packet based on this timing information, the decoding processing can be conducted at the timing specified by the time information PTS and the video data can be output at the timing in synchronism with the audio data, the occurrence of excess or lack of data can be prevented.
Note that, in the case of video data, a horizontal synchronizing signal and a vertical synchronizing signal are needed when displaying video. These horizontal and vertical synchronizing signals are to be formed based on the timing relative to the time information PTS in the decoder 17.
In the conventional digital broadcasting system 1 constructed as described above, video and audio data for multiple programs can be transmitted. However, as to each program, only one video data can be transmitted. Even in one program, if a plurality of video data associated with the program can be transmitted, it is considered that the usability of the digital broadcasting system can be further improved. For example, in the case where the program A is a concert program of a prescribed musician, if the low angle video data obtained by photographing the concert from the lower part, the middle angle video data obtained by photographing the concert from the front, and the high angle video data obtained by photographing the concert from the upper part can be transmitted as the video data of the program A, the user can select desired video data among the plurality of video data of the program A to view the program that fits to his taste. Thus, it is considered that the usability can be further improved.
In order to transmit a plurality of video data for one program, the conventional digital broadcasting system 1 treats the plurality of video data which are originally one program, as separate programs, and multiplexes and transmits them as the separate programs respectively. For example, if the video data of middle angle program A is transmitted as a program A, the video data of low angle program A is transmitted as a program B, and the video data of high angle program A is transmitted as a program C, desired video data can be selected according to the principle of program selection described above. However, if these were treated merely as the separate programs, when shifting the video data from middle angle to high angle video data, it is necessary to obtain the program information PMT whenever the video data are shifted because these are different programs. Therefore, the reception processing becomes complicated, and there is a possibility that it cannot be shifted to another video data immediately. Moreover, in the case where the plurality of data are treated as separate programs, programs cannot be synchronized with each other. Thus, there is a possibility that video data does not continue when the video data is shifted.
In view of the foregoing, an object of this invention is to provide a video transmission device and its method capable of transmitting a plurality of video data associated with one program without a complicated reception processing.
The foregoing object and other objects of the invention have been achieved by the provision of a video transmission device and its method. In the present invention, a plurality of associated video data is encoded respectively and then, different packet identification information is added to each coded video data to packetize the coded video data. And in order to control the plurality of associated video data as one program, first program information to show the packet identification information added to coded video data respectively, per video data is formed, and proper packet identification information is added to the first program information to packetize the first program information. Then, by multiplexing packets concerning each video data and the first program information, first transmission data for one program is formed. Further, the first transmission data, the second transmission data formed of a different program, and second program information to show the packet identification information added to the first program information of each program to be transmitted, per program are multiplexed in order to form and transmit third transmission data through the predetermined circuit.
In this way, in the case of transmitting a plurality of associated video data, since the first program information for controlling the plurality of associated video data as one program is formed and transmitted, video data can be shifted without receiving program information whenever shifting video data, like the conventional case in which associated video data are transmitted as separate programs. Thus, the processing of the receiving side can be simplified.
The nature, principle and utility of the invention will become apparent from the following detailed description when read in conjunction with the accompanying drawings in which like parts are designated by like reference numerals or characters.